Mixed-quantum-state detection with inconclusive results

We consider the problem of designing an optimal quantum detector with a fixed rate of inconclusive results that maximizes the probability of correct detection, when distinguishing between a collection of mixed quantum states. We show that the design of the optimal detector can be formulated as a semidefinite programming problem, and derive a set of necessary and sufficient conditions for an optimal measurement. We then develop a sufficient condition for the scaled inverse measurement to maximize the probability of correct detection for the case in which the rate of inconclusive results exceeds a certain threshold. Using this condition we derive the optimal measurement for linearly independent pure-state sets, and for mixed-state sets with a broad class of symmetries. Specifically, we consider geometrically uniform ~GU! state sets and compound geometrically uniform ~CGU! state sets with generators that satisfy a certain constraint. We then show that the optimal measurements corresponding to GU and CGU state sets with arbitrary generators are also GU and CGU, respectively, with generators that can be computed very efficiently in polynomial time within any desired accuracy by solving a reduced size semidefinite programming problem.